With various Network Monitoring Systems (NMS) in place, computer networks are steadily
becoming more resilient while also optimizing their performance and availability. Such
systems are often part of domains that fall under certain administrative policies and
access to monitoring data is limited to the domain's administrators and users. Making
it available to third-parties will help approach problems such as jitter, packet loss,
transmission delay and bandwidth availability amongst others on a multi-domain level.
However, the vast amount of network technologies, policies and managers turn its
distribution into a complicated process. Such restrictions have led to initiatives for
the development of Network Monitoring Service-Oriented Architectures
(NMSOA)\cite{NMSOA}. A conjoint effort between educational entities and National
Research Networks (NRENs), the perfSONAR framework is one initiative that enables the
discovery, collection, storage and distribution of monitoring data. At its heart, the
framework strives to diminish all data access-related administrative restrictions and
make it easier to troubleshoot end-to-end performance problems on multi-domain paths.
The perfSONAR framework originated out of the following three 
contexts\footnote{http://www.perfsonar.net/}:

\begin{itemize} 
\item A consortium of organizations who seek to build network performance middleware 
that is interoperable across multiple networks and useful for intra- and inter-network 
analysis. One of the main goals is to make it easier to solve end-to-end performance 
problems on paths crossing several networks.

\item A protocol. It assumes a set of roles (the various service
types), defines the protocol standard (syntax and semantics) by which they
communicate, and allows anyone to write a service playing one of those roles. The
protocol is based on SOAP XML messages and following the Open Grid Forum (OGF) Network
Measurement Working Group (NM-WG).

\item Several, interoperable software packages (implementations of various services) 
that attempt to implement an interoperable performance middleware framework. Those 
sets of code are developed by different partners.
\end{itemize}

\newpage

\section{Architecture of the framework}
perfSONAR defines a decentralized system for sharing network measurements. The general
monitoring infrastructure implemented by the framework is depicted in
Fig.~\ref{fig:perfsonarDistributedMiddleware}.

\begin{figure}[h]
	\begin{center}$
		\begin{array}{cc}
			\includegraphics[width=1.6in]{Figures/PSmiddlewareAPI.png} &
			\includegraphics[width=2.9in]{Figures/perfsonarDistributedMiddleware.jpg}
		\end{array}$
	\end{center}
	\caption{The three layered infrastructure of the perfSONAR framework.}
	\label{fig:perfsonarDistributedMiddleware}
\end{figure}
\vskip2em

The base layer defines domain-specific Measurement Points (MP) capable of collecting
distinct metrics obtained by an underlying monitoring tool. Multiple MPs can be
deployed within a domain. The Services Layer is the middle layer of the system and can
span across multiple administrative domains.  It enables the inter-domain exchange
of measurement data and management information through disjoint Web Services (WS). The
User Interface Layer consists of reporting and visualization tools that can present
data in customizable ways.

\subsection{Measurement Point Layer}
At this layer, the tasks of obtaining and storing measurement data are defined. The
measurements themselves are carried out by active or passive monitoring tools such as
Ping, Iperf or SNMP queries. Each MP is designed as a wrapper around a particular
monitoring tool while also defining interfaces for remote invocations. In principle,
MPs are highly versatile and do not pose any restriction on what technology or
programming language is used for defining them as long as their communication with the
rest of the framework is realized in accordance with the communication standard. The
current perfSONAR releases provide MPs capable of interfacing with the following
resources and measurement tools:

\begin{itemize}
	\item Telnet/SSH MP - returns the output of "traceroute" and "ping" commands
	\item Round-Robin Databases (RRD) MP
	\item Structured Query Language (SQL) Database MP
	\item Pinger MP - ICMP ping command wrapper
	\item Bandwidth Control (BWCTL) MP - Iperf TCP/UDP throughput measurement tool wrapper
\end{itemize}

\subsection{Services Layer}
The Services Layer provides a high degree of access transparency between data
collectors and data consumers by diminishing differences in data access and
representation. It does so by defining the following services:

\begin{itemize}
	\item Authentication and Authorization - Authentication Service (AS) 
	\item Discovery of services alike in other domains - Lookup Service (LS)
	\item Aggregation, correlation and filtering of measurement data - Transformation Service (TS)
	\item Storage of measurement data collected by MPs - Measurement Archives (MA)
\end{itemize}

The interaction of entities part of the layer as well as access to the Measurement
Point layer is not visible to the user. For this to be achieved, all data providers
define a "publisher" interface while all data consumers implement a "subscriber"
interface.

\subsection{User Interface Layer}
The User Interface Layer consists of entities such as visualization tools (GUIs) that
enable end-users to adapt, format and visualize measurement data in versatile ways
with the needs of certain applications and user groups in mind. Users of any service, 
regardless of whether they are end-user application or services themselves are regarded 
as clients. These services allow users to perform tests using the lower layers of the 
framework. From the perspective of users, the Service Layer abstracts the differences of
Measurement Points deployed across the different participating domains.

\subsection{NMWG protocol}
The data communication protocol used by the framework's services is based on SOAP
(Simple Object Access Protocol) XML messages and adheres to the Global Grid Forum
(GGF) Network Monitoring Working Group (NMWG) XML schema. HTTP is used as the
underlying transport protocol. The general structure of messages exchanged between
services is depicted in Fig.~\ref{fig:perfSONAR_transport_protocol} 

\begin{figure}[h]
	\begin{center}
			\includegraphics[width=2.8in]{Figures/perfSONAR_transport_protocol.jpg}
	\end{center}
	\caption{The encapsulation of metadata + data in the NMWG protocol.}
	\label{fig:perfSONAR_transport_protocol}
\end{figure}
\vskip2em

The NMWG schema is also used for normalizing measurement data. This is achieved by
segmenting the presentation of measurement data into two parts: meta data and data.
The paradigm allows for easily extending the data types supported by the framework's
communication protocol when new MPs and MAs are deployed within the existing
infrastructure.

\section{Implementations}
Two perfSONAR implementation exist at current that fully comply with the NMSOA
depicted in Fig.~\ref{fig:perfsonarDistributedMiddleware}, namely perfSONAR-PS and
perfSONAR MDM.  They are actively developed by different NRENs for servicing the
requirements of particular users bases and infrastructures. perfSONAR MDM is developed
by GÉANT and aims at providing a seamless Multi-Domain Monitoring system for the GÉANT
Service Area. Measurement Points and client services provided by this implementation
are developed with the needs of NOC and PERT engineers in mind. perfSONAR-PS is
developed by ESnet and Internet2 with the purpose of optimizing the troubleshooting
process between campus and research networks as well as network providers. 
\newpage
A third implementation (perfSONAR NC) developed by UNINET offers standalone
Measurement Points incapable of registering with a local or global lookup service. An
overview of available perfSONAR implementations is depicted in
Table~\ref{tab:perfSONAR_releases_comparison}

\begin{table}[h]
\begin{center}
  \begin{tabular}{ | l | l | l | p{3cm} |}
    \hline
    \textbf{Feature}           & \textbf{perfSONAR-PS/MDM}          & \textbf{perfSONAR NC}                            \\ \hline
    Services                   & MP, MA, LS,  AS, TS                & MP                                               \\ \hline
    Protocol                   & NMWG                               & NETCONF                                          \\ \hline
    Data models                & none                               & YANG                                             \\ \hline
    Query Mechanisms           & MA/MP dependent                    & XPATH                                            \\ \hline
    Scalability                & ~1000 domains                      & >200K dinaubs                                    \\ \hline
    Code base                  & >6000 lines, MA specific ~3000     & <1000 lines,                                     \\ 
    ~                          &                                    & MA specific <200                                 \\ \hline
    Programming language       & Perl/Java                          & PHP                                              \\
    \hline
  \end{tabular}
\end{center}
	\caption{perfSONAR features by release}
	\label{tab:perfSONAR_releases_comparison}
\end{table}


The major differences between the perfSONAR-PS and MDM releases is the programming
language used in their implementation as well as the type of Measurement Points they
define. Measurement Points from both releases (Table~\ref{tab:perfSONAR_MAs}) that
target the distribution of the same metric are still capable of communicating with one
another because of the standardized NMWG protocol. Both releases do not follow a
certain data model through which services at the lower two layers of the framework
need to be implemented. Therefore, the query mechanisms that need to be used with a
given service are highly specific to its own interfaces. A complete comparison of the
main characteristics that distinguish the releases from one another can be found in
\cite{perfSONAR_MDM_PS_comparison}

\begin{table}[h]
\begin{center}
  \begin{tabular}{ | l | c | c | }
    \hline
    \textbf{Measurement Archive} & \textbf{perfSONAR-PS} & \textbf{perfSONAR MDM} \\ \hline
    Reverse Traceroute           & \textbullet               & \textbullet                \\ \hline
    Reverse Ping                 & \textbullet               & \textbullet                \\ \hline
    BWCTL(Iperf, NutTCP)         & \textbullet               & \textbullet                \\ \hline
    OWAMP (RFC4656)              & \textbullet               & \textbullet                \\ \hline
    SNMP Round-Robin Database    & \textbullet               & \textbullet                \\ \hline
    SQL database                 &                           & \textbullet                \\
    \hline
  \end{tabular}
\end{center}
	\caption{perfSONAR-PS/MDM Measurement Archives}
	\label{tab:perfSONAR_MAs}
\end{table}

\section{Extensibility}
To instrument the software towards the distribution of new metrics, a deep
familiarization with the framework was conducted.  The outcome showed that the NMSOA
framework adopted by the perfSONAR-PS and MDM distributions is a good choice for a
system, which to instrument towards energy profiling of network nodes for the
following reasons:

\begin{itemize}
	\item The software provides a uniform, well-organized system for the collection, distribution and consumption of network-related metrics that mediates
	\begin{itemize}
		\item data discovery, and
		\item data access 
	\end{itemize}
	\item The high degree of access transparency provided by the Service layer
		  enables the seamless integration of newly implemented Measurement Archive in perfSONARs current global infrastructure.
	\item Instrumenting the software towards the distribution of new metrics can be achieved in an autonomous way by:
	\begin{itemize}
		\item Provisioning of new NMWG schemas that describes the underlying measurement data
		\item Provisioning of new Measurement Points and Archives that collect, store and serve the data.
	\end{itemize}
	\item Its user base is comprised of various NRENs, which makes it a perfect testbed for such a new and unstandardized system
\end{itemize}


\subsection{Code base extensibility}
In order to evaluate what are the important parts of creating a new Measurement
Archive, an extensive examination of the code base of perfSONAR-PS was conducted as
the software is a community-driven, agile-programming~\cite{perfSONAR_release_management} reference to perfSONAR
release management) effort, which lacks a standardized documentation base. Development
is mainly carried out via conference calls and mailing lists~\cite{perfSONAR_MDM_PS_comparison}. 


With a lack of documentation on the subscriber/publisher APIs defined by the different
services, determining how to create a new Measurement Archive relied on performing a
full installation of the perfSONAR-PS with the purpose of first verifying the valid
functionality of the different global and local services. Afterwards, the source code
bundles of the different services were examined. The following research questions were
defined:

\begin{enumerate}
	\item How do Measurement Archives register and maintain communication with local and global lookup services?
	\item Is there a generic data model followed in the implementation of Measurement Points and Archives?
	\item How is data collected by a Measurement Point
	\begin{enumerate}[label=\alph*.]
		\item Normalized
		\item Made available to a Measurement Archive
	\end{enumerate}
	\item Is there any stateful control information kept and how is it represented in terms of the NMWG XML schema?
\end{enumerate}

Due to the extensive code base of the software the only question that was concisely
answered is 3a. A Measurement Point service wraps around existing monitoring tools by
either directly invoking them or by relying on storage containers in which such tools
first write collected data. Data is then normalized with regards to the NMWG XML
schema. The Measurement Point service also defines mechanisms for pushing data towards
Measurement Archives. However, the different implementation of Measurement Archives do
not interpret the entire NMWG schema space, but rather define hard-coded routines
through which underlying measurement data is referenced to subspaces of the schema. As
a result, services such as the SNMP Measurement Archive cannot interpret objects from
the entire SNMP OID tree but rather target the exposure of interface statistics alone
through archive-specific query routines. Likewise, RRD Measurement Archives do not
treat RRD databases in a generic fashion by first building a profile of the underlying
Round-Robin Archives of the database and normalizing it in the scope of an NMWG
measurement schema.

In order to answer the remaining questions, contact was established with one of
ESnet's perfSONAR-PS developers.  After sharing the purpose of our research and the
concerns in terms of extensibility, it was established that implementing a new
Measurement Archive is not feasible within the time limitations of the project.
